Papers with attention component
Overcoming the Rare Word Problem for low-resource language pairs in Neural Machine Translation (D19-52)
Copied to clipboard
| Challenge: | Despite above approaches can improve the prediction of rare words, they still have challenges which have adverse effects on its effectiveness. |
| Approach: | They propose three ways to address rare-word problem in neural machine translation systems . they propose an algorithm to learn morphology of unknown words for English in supervised way to minimize adverse effect of rare- word problem. |
| Outcome: | The proposed approaches improve accuracy on two low-resource language pairs. |
Neural Hidden Markov Model for Machine Translation (P18-2)
Copied to clipboard
| Challenge: | Attention-based neural machine translation models selectively focus on specific source positions to produce a translation. |
| Approach: | They propose to replace the attention component with a neural hidden Markov model that selectively focuss on specific source positions to produce a translation. |
| Outcome: | The proposed model performs better than the state-of-the-art attention-based models on the GermanEnglish and ChineseEnglish translation tasks. |
Refining Source Representations with Relation Networks for Neural Machine Translation (C18-1)
Copied to clipboard
| Challenge: | Existing neural machine translation frameworks that forget distant information and disregard relationship between source and target words are not effective. |
| Approach: | They propose to use relation networks to learn better representations of the source . they propose to associate source words with each other to help retain their relationships . |
| Outcome: | Experiments show that the proposed approach outperforms the encoder-decoder framework on several datasets. |
Multi-Domain Dialogue State Tracking with Disentangled Domain-Slot Attention (2023.findings-acl)
Copied to clipboard
| Challenge: | Multi-domain dialogue state tracking is a challenge for task-oriented dialogue systems . domains and slots are aggregated into a single query to generate domain-slot specific representations . |
| Approach: | They propose to disentangle domain-slot attention for multi-domain dialogue state tracking by separating query about domains and slots from the attention component. |
| Outcome: | The proposed approach outperforms the standard multi-head attention with aggregated domain-slot query. |
PaLM: A Hybrid Parser and Language Model (D19-1)
Copied to clipboard
| Challenge: | Recent language models have shown strong data-fitting performance, but do not explicitly encode any notion of structural information. |
| Approach: | They propose a hybrid parser and neural language model that adds an attention layer over text spans in the left context. |
| Outcome: | The proposed model outperforms baseline models on language modeling and provides syntactically-informed representations of the context. |